%%\input{epsf.tex}
\chapter{\label{scott}Linux Interprocess Communications}
\author{B. Scott Burkett, {\tt scottb@intnet.net}}
\date{v1.0, 29 March 1995}
\abstract{A detailed overview of the IPC (interprocess communication facilities) facilities
implemented in the Linux Operating System.}


\section{Introduction}

The Linux IPC (Inter-process communication) facilities provide a method for
multiple processes to communicate with one another.  There are several
methods of IPC available to Linux C programmers:
\begin{itemize}
\item Half-duplex UNIX pipes
\item FIFOs (named pipes)
\item SYSV style message queues
\item SYSV style semaphore sets
\item SYSV style shared memory segments
\item Networking sockets (Berkeley style) (not covered in this paper)
\item Full-duplex pipes (STREAMS pipes) (not covered in this paper)
\end{itemize}

These facilities, when used effectively, provide a solid framework for
client/server development on any UNIX system (including Linux).




\section{Half-duplex UNIX Pipes}




\subsection{Basic Concepts}

Simply put, a {\em pipe\/} is a method of connecting the {\em standard output\/} of one
process to the {\em standard input\/} of another.  Pipes are the eldest of the IPC
tools, having been around since the earliest incarnations of the UNIX
operating system.  They provide a method of one-way communications (hence
the term half-duplex) between processes.

This feature is widely used, even on the UNIX command line (in the shell).

\begin{tscreen}
\begin{verbatim}
        ls | sort | lp
\end{verbatim}


\end{tscreen}


The above sets up a pipeline, taking the output of ls as the input of 
sort, and the output of sort as the input of lp.  The data is running through a
half duplex pipe, traveling (visually) left to right through the pipeline.

Although most of us use pipes quite religiously in shell script programming, 
we often do so without giving a second thought to what transpires at the kernel
level. 

When a process creates a pipe, the kernel sets up two file descriptors for use
by the pipe.  One descriptor is used to allow a path of input into the pipe (write),
while the other is used to obtain data from the pipe (read).  At this point, the
pipe is of little practical use, as the creating process can only use the pipe to
communicate with itself.  Consider this representation of a process and the
kernel after a pipe has been created:

\begin {center}
\input {pipe1}
\end   {center}

From the above diagram, it is easy to see how the descriptors are connected
together.  If the process sends data through the pipe (fd0), it has the ability
to obtain (read) that information from fd1.  However, there is a much larger
objective of the simplistic sketch above.  While a pipe initially connects a 
process to itself, data traveling through the pipe moves through the kernel.
Under Linux, in particular, pipes are actually represented internally with a
valid inode.  Of course, this inode resides within the kernel itself, and not
within the bounds of any physical file system.  This particular point will
open up some pretty handy I/O doors for us, as we will see a bit later on.

At this point, the pipe is fairly useless.  After all, why go to the trouble of
creating a pipe if we are only going to talk to ourself?  At this point, the 
creating process typically forks a child process.  Since a child process will
inherit any open file descriptors from the parent, we now have the basis for
multiprocess communication (between parent and child).  Consider this updated
version of our simple sketch:

\begin {center}
\input {pipe2}
\end   {center}

Above, we see that both processes now have access to the file descriptors which
constitute the pipeline.  It is at this stage, that a critical decision must be
made.  In which direction do we desire data to travel?  Does the child process
send information to the parent, or vice-versa?  The two processes mutually agree
on this issue, and proceed to ``close'' the end of the pipe that they are not
concerned with.  For discussion purposes, let's say the child performs some
processing, and sends information back through the pipe to the parent.  Our
newly revised sketch would appear as such:

\begin {center}
\input {pipe3}
\end   {center}

Construction of the pipeline is now complete!  The only thing left to do is
make use of the pipe.  To access a pipe directly, the same system calls that
are used for low-level file I/O can be used (recall that pipes are actually
represented internally as a valid inode).

To send data to the pipe, we use the write() system call, and to retrieve data
from the pipe, we use the read() system call.  Remember, low-level file I/O
system calls work with file descriptors!  However, keep in mind that certain
system calls, such as lseek(), do not work with descriptors to pipes.




\subsection{Creating Pipes in C}



Creating ``pipelines'' with the C programming language can be a bit more involved
than our simple shell example.  To create a simple pipe with C, we make use of
the pipe() system call.  It takes a single argument, which is an array of two
integers, and if successful, the array will contain two new file descriptors
to be used for the pipeline. After creating a pipe, the process typically
spawns a new process (remember the child inherits open file descriptors).



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: pipe();                                                          

  PROTOTYPE: int pipe( int fd[2] );                                             
    RETURNS: 0 on success                                                       
             -1 on error: errno = EMFILE (no free descriptors)                  
                                  EMFILE (system file table is full)            
                                  EFAULT (fd array is not valid)                

  NOTES: fd[0] is set up for reading, fd[1] is set up for writing               
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The first integer in the array (element 0) is set up and opened for reading,
while the second integer (element 1) is set up and opened for writing.  
Visually speaking, the output of fd$1$ becomes the input for fd$0$.  Once
again, all data traveling through the pipe moves through the kernel.

\begin{verbatim}
        #include <stdio.h>
        #include <unistd.h>
        #include <sys/types.h>

        main()
        {
                int     fd[2];
                
                pipe(fd);
                .
                .
        }
\end{verbatim}




Remember that an array name in C {\em decays\/} into a pointer to its first
member.  Above, {\tt fd} is equivalent to {\tt \&fd[0]}.  Once we have established the
pipeline, we then fork our new child process:

\begin{verbatim}
        #include <stdio.h>
        #include <unistd.h>
        #include <sys/types.h>

        main()
        {
                int     fd[2];
                pid_t   childpid;

                pipe(fd);
        
                if((childpid = fork()) == -1)
                {
                        perror("fork");
                        exit(1);
                }
                .
                .
        }
\end{verbatim}




If the parent wants to receive data from the child, it should close fd$1$,
and the child should close fd$0$. If the parent wants to send data to the
child, it should close fd$0$, and the child should close fd$1$. Since
descriptors are shared between the parent and child, we should always be sure
to close the end of pipe we aren't concerned with.  On a technical note, the
EOF will never be returned if the unnecessary ends of the pipe are not
explicitly closed.

\begin{verbatim}
        #include <stdio.h>
        #include <unistd.h>
        #include <sys/types.h>

        main()
        {
                int     fd[2];
                pid_t   childpid;

                pipe(fd);
        
                if((childpid = fork()) == -1)
                {
                        perror("fork");
                        exit(1);
                }

                if(childpid == 0)
                {
                        /* Child process closes up input side of pipe */
                        close(fd[0]);
                }
                else
                {
                        /* Parent process closes up output side of pipe */
                        close(fd[1]);
                }
                .
                .
        }
\end{verbatim}




As mentioned previously, once the pipeline has been established, the file
descriptors may be treated like descriptors to normal files.

\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: pipe.c
 *****************************************************************************/

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>

int main(void)
{
        int     fd[2], nbytes;
        pid_t   childpid;
        char    string[] = "Hello, world!\n";
        char    readbuffer[80];

        pipe(fd);
        
        if((childpid = fork()) == -1)
        {
                perror("fork");
                exit(1);
        }

        if(childpid == 0)
        {
                /* Child process closes up input side of pipe */
                close(fd[0]);

                /* Send "string" through the output side of pipe */
                write(fd[1], string, strlen(string));
                exit(0);
        }
        else
        {
                /* Parent process closes up output side of pipe */
                close(fd[1]);

                /* Read in a string from the pipe */
                nbytes = read(fd[0], readbuffer, sizeof(readbuffer));
                printf("Received string: %s", readbuffer);
        }
        
        return(0);
}
\end{verbatim}




Often, the descriptors in the child are duplicated onto standard input 
or output.  The child can then exec() another program, which inherits the
standard streams.  Let's look at the dup() system call:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: dup();                                                           

  PROTOTYPE: int dup( int oldfd );                                              
    RETURNS: new descriptor on success                                          
             -1 on error: errno = EBADF (oldfd is not a valid descriptor)       
                                  EBADF (newfd is out of range)                 
                                  EMFILE (too many descriptors for the process) 

  NOTES: the old descriptor is not closed!  Both may be used interchangeably    
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}






Although the old descriptor and the newly created descriptor can be used
interchangeably, we will typically close one of the standard streams first.
The dup() system call uses the lowest-numbered, unused descriptor for the
new one.

Consider:

\begin{verbatim}
        .
        .
        childpid = fork();
        
        if(childpid == 0)
        {
                /* Close up standard input of the child */
                close(0);
                
                /* Duplicate the input side of pipe to stdin */
                dup(fd[0]);
                execlp("sort", "sort", NULL);
                .
        }
\end{verbatim}




Since file descriptor 0 (stdin) was closed, the call to dup() duplicated the 
input descriptor of the pipe (fd$0$) onto its standard input.  We then make
a call to execlp(), to overlay the child's text segment (code) with that of
the sort program.  Since newly exec'd programs inherit standard streams from
their spawners, it actually inherits the input side of the pipe as its
standard input!  Now, anything that the original parent process sends to the
pipe, goes into the sort facility.

There is another system call, dup2(), which can be used as well.  This 
particular call originated with Version 7 of UNIX, and was carried on through
the BSD releases and is now required by the POSIX standard. 



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: dup2();                                                          

  PROTOTYPE: int dup2( int oldfd, int newfd );                                  
    RETURNS: new descriptor on success                                          
             -1 on error: errno = EBADF (oldfd is not a valid descriptor)       
                                  EBADF (newfd is out of range)                 
                                  EMFILE (too many descriptors for the process) 

  NOTES: the old descriptor is closed with dup2()!                              
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




With this particular call, we have the close operation, and the actual
descriptor duplication, wrapped up in one system call.  In addition, it
is guaranteed to be atomic, which essentially means that it will never
be interrupted by an arriving signal.  The entire operation will transpire
before returning control to the kernel for signal dispatching.  With the
original dup() system call, programmers had to perform a close() operation
before calling it.  That resulted in two system calls, with a small degree
of vulnerability in the brief amount of time which elapsed between them.  
If a signal arrived during that brief instance, the descriptor duplication
would fail.  Of course, dup2() solves this problem for us.

Consider:

\begin{verbatim}
        .
        .
        childpid = fork();
        
        if(childpid == 0)
        {
                /* Close stdin, duplicate the input side of pipe to stdin */
                dup2(0, fd[0]);
                execlp("sort", "sort", NULL);
                .
                .
        }
\end{verbatim}







\subsection{Pipes the Easy Way!}



If all of the above ramblings seem like a very round-about way of creating
and utilizing pipes, there is an alternative.  



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  LIBRARY FUNCTION: popen();                                                    

  PROTOTYPE: FILE *popen ( char *command, char *type);                          
    RETURNS: new file stream on success                                         
             NULL on unsuccessful fork() or pipe() call                         

  NOTES: creates a pipe, and performs fork/exec operations using "command"      
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




This standard library function creates a half-duplex pipeline by calling
pipe() internally.  It then forks a child process, execs the Bourne shell,
and executes the "command" argument within the shell.  Direction of data
flow is determined by the second argument, "type".  It can be "r" or "w", for
"read" or "write".  It cannot be both!  Under Linux, the pipe will be opened
up in the mode specified by the first character of the "type" argument.  So,
if you try to pass "rw", it will only open it up in "read" mode.

While this library function performs quite a bit of the dirty work for you, 
there is a substantial tradeoff.  You lose the fine control you once had
by using the pipe() system call, and handling the fork/exec yourself.  However,
since the Bourne shell is used directly, shell metacharacter expansion
(including wildcards) is permissible within the "command" argument.

Pipes which are created with popen() must be closed with pclose().  By now,
you have probably realized that popen/pclose share a striking resemblance
to the standard file stream I/O functions fopen() and fclose().



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  LIBRARY FUNCTION: pclose();                                                   

  PROTOTYPE: int pclose( FILE *stream );                                        
    RETURNS: exit status of wait4() call                                        
             -1 if "stream" is not valid, or if wait4() fails                   

  NOTES: waits on the pipe process to terminate, then closes the stream.        
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The pclose() function performs a wait4() on the process forked by popen().
When it returns, it destroys the pipe and the file stream.  Once again, it is
synonymous with the fclose() function for normal stream-based file I/O.

Consider this example, which opens up a pipe to the sort command, and proceeds
to sort an array of strings:

\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: popen1.c
 *****************************************************************************/
  
#include <stdio.h>

#define MAXSTRS 5

int main(void)
{
        int  cntr;
        FILE *pipe_fp;
        char *strings[MAXSTRS] = { "echo", "bravo", "alpha",
                                  "charlie", "delta"};

        /* Create one way pipe line with call to popen() */
        if (( pipe_fp = popen("sort", "w")) == NULL)
        {
                perror("popen");
                exit(1);
        }

        /* Processing loop */
        for(cntr=0; cntr<MAXSTRS; cntr++) {
                fputs(strings[cntr], pipe_fp);
                fputc('\n', pipe_fp);
        }

        /* Close the pipe */
        pclose(pipe_fp);
        
        return(0);
}
\end{verbatim}




Since {\tt popen()} uses the shell to do its bidding, all shell expansion
characters and metacharacters are available for use!  In addition, more  
advanced techniques such as redirection, and even output piping, can be utilized
with {\tt popen()}.  Consider the following sample calls:

\begin{verbatim}
        popen("ls ~scottb", "r");
        popen("sort > /tmp/foo", "w");
        popen("sort | uniq | more", "w");
\end{verbatim}




As another example of popen(), consider this small program, which opens
up two pipes (one to the ls command, the other to sort):

\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: popen2.c
 *****************************************************************************/

#include <stdio.h>

int main(void)
{
        FILE *pipein_fp, *pipeout_fp;
        char readbuf[80];

        /* Create one way pipe line with call to popen() */
        if (( pipein_fp = popen("ls", "r")) == NULL)
        {
                perror("popen");
                exit(1);
        }

        /* Create one way pipe line with call to popen() */
        if (( pipeout_fp = popen("sort", "w")) == NULL)
        {
                perror("popen");
                exit(1);
        }

        /* Processing loop */
        while(fgets(readbuf, 80, pipein_fp))
                fputs(readbuf, pipeout_fp);

        /* Close the pipes */
        pclose(pipein_fp);
        pclose(pipeout_fp);

        return(0);
}
\end{verbatim}




For our final demonstration of popen(), let's create a generic program that
opens up a pipeline between a passed command and filename:

\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: popen3.c
 *****************************************************************************/

#include <stdio.h>

int main(int argc, char *argv[])
{
        FILE *pipe_fp, *infile;
        char readbuf[80];

        if( argc != 3) {
                fprintf(stderr, "USAGE:  popen3 [command] [filename]\n");       
                exit(1);
        }

        /* Open up input file */
        if (( infile = fopen(argv[2], "rt")) == NULL)
        {
                perror("fopen");
                exit(1);        
        }

        /* Create one way pipe line with call to popen() */
        if (( pipe_fp = popen(argv[1], "w")) == NULL)
        {
                perror("popen");
                exit(1);
        }

        /* Processing loop */
        do { 
                fgets(readbuf, 80, infile);
                if(feof(infile)) break;

                fputs(readbuf, pipe_fp);
        } while(!feof(infile));

        fclose(infile); 
        pclose(pipe_fp);

        return(0);
}
\end{verbatim}




Try this program out, with the following invocations:
\begin{verbatim}
        popen3 sort popen3.c
        popen3 cat popen3.c
        popen3 more popen3.c
        popen3 cat popen3.c | grep main
\end{verbatim}







\subsection{Atomic Operations with Pipes}



In order for an operation to be considered ``atomic'', it must not be interrupted
for any reason at all.  The entire operation occurs at once.  The POSIX 
standard dictates in /usr/include/posix1\_lim.h that the maximum buffer size
for an atomic operation on a pipe is:

\begin{verbatim}
        #define _POSIX_PIPE_BUF         512
\end{verbatim}




Up to 512 bytes can be written or retrieved from a pipe atomically.  Anything
that crosses this threshold will be split, and not atomic.  Under Linux,  
however, the atomic operational limit is defined in ``linux/limits.h'' as:

\begin{verbatim}
        #define PIPE_BUF        4096
\end{verbatim}




As you can see, Linux accommodates the minimum number of bytes required by
POSIX, quite considerably I might add.  The atomicity of a pipe operation
becomes important when more than one process is involved (FIFOS).  For
example, if the number of bytes written to a pipe exceeds the atomic limit
for a single operation, and multiple processes are writing to the pipe,
the data will be ``interleaved'' or ``chunked''.  In other words, one process
may insert data into the pipeline between the writes of another.




\subsection{Notes on half-duplex pipes:}



\begin{itemize}
\item Two way pipes can be created by opening up two pipes, and properly
reassigning the file descriptors in the child process.
\item The pipe() call must be made BEFORE a call to fork(), or the descriptors 
will not be inherited by the child!  (same for popen()).
\item With half-duplex pipes, any connected processes must share a related
ancestry.  Since the pipe resides within the confines of the kernel, any
process that is not in the ancestry for the creator of the pipe has no
way of addressing it.  This is not the case with named pipes (FIFOS).
\end{itemize}





\section{Named Pipes (FIFOs - First In First Out)}




\subsection{Basic Concepts}

A named pipe works much like a regular pipe, but does have some noticeable
differences.

\begin{itemize}
\item Named pipes exist as a device special file in the file system.
\item Processes of different ancestry can share data through a named pipe.
\item When all I/O is done by sharing processes, the named pipe remains in
the file system for later use.
\end{itemize}





\subsection{Creating a FIFO}



There are several ways of creating a named pipe.  The first two can be done
directly from the shell.

\begin{verbatim}
        mknod MYFIFO p
        mkfifo a=rw MYFIFO
\end{verbatim}




The above two commands perform identical operations, with one exception.  The
mkfifo command provides a hook for altering the permissions on the FIFO file
directly after creation.  With mknod, a quick call to the chmod command will
be necessary.

FIFO files can be quickly identified in a physical file system by the ``p''
indicator seen here in a long directory listing:

\begin{verbatim}
        $ ls -l MYFIFO
        prw-r--r--   1 root     root            0 Dec 14 22:15 MYFIFO|
\end{verbatim}




Also notice the vertical bar (``pipe sign'') located directly after the file
name.  Another great reason to run Linux, eh?

To create a FIFO in C, we can make use of the mknod() system call:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  LIBRARY FUNCTION: mknod();                                                    

  PROTOTYPE: int mknod( char *pathname, mode_t mode, dev_t dev);                
    RETURNS: 0 on success,                                                      
             -1 on error: errno = EFAULT (pathname invalid)                     
                                  EACCES (permission denied)                    
                                  ENAMETOOLONG (pathname too long)              
                                  ENOENT (invalid pathname)                     
                                  ENOTDIR (invalid pathname)                    
                                  (see man page for mknod for others)           

  NOTES: Creates a filesystem node (file, device file, or FIFO)                 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




I will leave a more detailed discussion of mknod() to the man page, but let's
consider a simple example of FIFO creation from C:

\begin{verbatim}
                mknod("/tmp/MYFIFO", S_IFIFO|0666, 0);
\end{verbatim}




In this case, the file ``/tmp/MYFIFO'' is created as a FIFO file.  The requested
permissions are ``0666'', although they are affected by the umask setting as  
follows:

\begin{verbatim}
                final_umask = requested_permissions & ~original_umask
\end{verbatim}




A common trick is to use the umask() system call to temporarily zap the 
umask value:

\begin{verbatim}
                umask(0);
                mknod("/tmp/MYFIFO", S_IFIFO|0666, 0);
\end{verbatim}




In addition, the third argument to mknod() is ignored unless we are creating
a device file.  In that instance, it should specify the major and minor
numbers of the device file.




\subsection{FIFO Operations}



I/O operations on a FIFO are essentially the same as for normal pipes, with
once major exception.  An ``open'' system call or library function should be 
used to physically open up a channel to the pipe.  With half-duplex pipes,
this is unnecessary, since the pipe resides in the kernel and not on a physical
filesystem.  In our examples, we will treat the pipe as a stream, opening it
up with fopen(), and closing it with fclose().

Consider a simple server process:

\begin{verbatim}
/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: fifoserver.c
 *****************************************************************************/

#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <unistd.h>

#include <linux/stat.h>

#define FIFO_FILE       "MYFIFO"

int main(void)
{
        FILE *fp;
        char readbuf[80];

        /* Create the FIFO if it does not exist */
        umask(0);
        mknod(FIFO_FILE, S_IFIFO|0666, 0);

        while(1)
        {
                fp = fopen(FIFO_FILE, "r");
                fgets(readbuf, 80, fp);
                printf("Received string: %s\n", readbuf);
                fclose(fp);
        }

        return(0);
}
\end{verbatim}




Since a FIFO blocks by default, run the server in the background after you
compile it:

\begin{verbatim}
        $ fifoserver&
\end{verbatim}




We will discuss a FIFO's blocking action in a moment.  First, consider the
following simple client frontend to our server:

\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: fifoclient.c
 *****************************************************************************/

#include <stdio.h>
#include <stdlib.h>

#define FIFO_FILE       "MYFIFO"

int main(int argc, char *argv[])
{
        FILE *fp;

        if ( argc != 2 ) {
                printf("USAGE: fifoclient [string]\n");
                exit(1);
        }

        if((fp = fopen(FIFO_FILE, "w")) == NULL) {
                perror("fopen");
                exit(1);
        }

        fputs(argv[1], fp);

        fclose(fp);
        return(0);
}
\end{verbatim}







\subsection{Blocking Actions on a FIFO}



Normally, blocking occurs on a FIFO.  In other words, if the FIFO is opened
for reading, the process will "block" until some other process opens it for
writing.  This action works vice-versa as well.  If this behavior is 
undesirable, the O\_NONBLOCK flag can be used in an open() call to disable the
default blocking action.

In the case with our simple server, we just shoved it into the background,
and let it do its blocking there.  The alternative would be to jump to another
virtual console and run the client end, switching back and forth to see the
resulting action.




\subsection{The Infamous SIGPIPE Signal}



On a last note, pipes must have a reader and a writer.  If a process tries to
write to a pipe that has no reader, it will be sent the SIGPIPE signal from
the kernel.  This is imperative when more than two processes are involved in
a pipeline.










\section{System V IPC}






\subsection{Fundamental Concepts}

With System V, AT\&T introduced three new forms of IPC facilities (message
queues, semaphores, and shared memory).  While the POSIX committee has not yet
completed its standardization of these facilities, most implementations do
support these.  In addition, Berkeley (BSD) uses sockets as its primary form
of IPC, rather than the System V elements.  Linux has the ability to use 
both forms of IPC (BSD and System V), although we will not discuss sockets
until a later chapter.

The Linux implementation of System V IPC was authored by {\em Krishna Balasubramanian\/},
at {\tt balasub@cis.ohio-state.edu}.




\subsubsection{IPC Identifiers}

Each IPC {\em object\/} has a unique IPC identifier associated with it.  When
we say ``IPC object'', we are speaking of a single message queue, semaphore set,
or shared memory segment.
This identifier is used within the kernel to uniquely identify an IPC object.
For example, to access a particular shared memory segment, the only item you
need is the unique ID value which has been assigned to that segment.

The uniqueness of an identifier is relevant to the {\em type\/} of object
in question.  To illustrate this, assume a numeric identifier of ``12345''. 
While there can never be two message queues with this same identifier, there
exists the distinct possibility of a message queue and, say, a shared memory
segment, which have the same numeric identifier.




\subsubsection{IPC Keys}

To obtain a unique ID, a {\em key\/} must be used.  The key must be mutually
agreed upon by both client and server processes.  This represents the first
step in constructing a client/server framework for an application. 



{\em When you use a telephone to call someone,  you
must know their number.  In addition, the phone company must know how to relay
your outgoing call to its final destination.  Once the other party responds
by answering the telephone call, the connection is made.\/}



In the case of System V IPC facilities, the ``telephone'' correllates directly
with the type of object being used.  The ``phone company'', or routing method,
can be directly associated with an IPC key.

The key can be the same value every time, by hardcoding a key value into an
application.  This has the disadvantage of the key possibly being in use
already.  Often, the ftok() function is used to generate key values for both
the client and the server.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  LIBRARY FUNCTION: ftok();

  PROTOTYPE: key_t ftok ( char *pathname, char proj );
    RETURNS: new IPC key value if successful
             -1 if unsuccessful, errno set to return of stat() call
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The returned key value from ftok() is generated by combining the inode number
and minor device number from the file in argument one, with the one character
project indentifier in the second argument.  This doesn't guarantee uniqueness,
but an application can check for collisions and retry the key generation.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        key_t   mykey;
        mykey = ftok("/tmp/myapp", 'a');
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




In the above snippet, the directory {\tt /tmp/myapp} is combined with the
one letter identifier of {\tt 'a'}.  Another common example is to use the
current directory:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        key_t   mykey;
        mykey = ftok(".", 'a');
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The key generation algorithm used is completely up to the discretion of the
application programmer.  As long as measures are in place to prevent race
conditions, deadlocks, etc, any method is viable.  For our demonstration
purposes, we will use the ftok() approach.  If we assume that each client
process will be running from a unique ``home'' directory, the keys generated
should suffice for our needs.

The key value, however it is obtained, is used in subsequent IPC system calls
to create or gain access to IPC objects.




\subsubsection{The {\tt ipcs} Command}

The {\tt ipcs} command can be used to obtain the status of all System V IPC objects.
The Linux version of this tool was also authored by {\em Krishna Balasubramanian\/}.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
ipcs     -q:    Show only message queues
ipcs     -s:    Show only semaphores
ipcs     -m:    Show only shared memory
ipcs --help:    Additional arguments
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}


By default, all three categories of objects are shown.  Consider the following sample
output of {\tt ipcs}:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

------ Shared Memory Segments --------
shmid     owner     perms     bytes     nattch    status      

------ Semaphore Arrays --------
semid     owner     perms     nsems     status      

------ Message Queues --------
msqid     owner     perms     used-bytes  messages    
0         root      660       5           1           
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Here we see a single message queue which has an identifier of ``0''.  It is owned by 
the user {\em root\/}, and has octal permissions of {\tt 660}, or {\tt -rw-rw---}. 
There is one message in the queue, and that message has a total size of 5 bytes.

The {\tt ipcs} command is a very powerful tool which provides a peek into the kernel's
storage mechanisms for IPC objects.  Learn it, use it, revere it.




\subsubsection{The {\tt ipcrm} Command}

The {\tt ipcrm} command can be used to remove an IPC object from the kernel.  While IPC
objects can be removed via system calls in user code (we'll see how in a moment), the need
often arises, especially under development environments, to remove IPC objects manually.  Its
usage is simple:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
ipcrm <msg | sem | shm>  <IPC ID>
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Simply specify whether the object to be deleted is a message queue ({\em msg\/}), a semaphore
set ({\em sem\/}), or a shared memory segment ({\em shm\/}).  The IPC ID can be obtained by
the {\tt ipcs} command.  You have to specify the type of object, since identifiers are unique
among the same type (recall our discussion of this earlier).




\subsection{Message Queues}




\subsubsection{Basic Concepts}

Message queues can be best described as an internal linked list within the
kernel's addressing space.  Messages can be sent to the queue in order and
retrieved from the queue in several different ways.  Each message queue (of
course) is uniquely identified by an IPC identifier.




\subsubsection{Internal and User Data Structures}

The key to fully understanding such complex topics as System V IPC is to become
intimately familiar with the various internal data structures that reside within
the confines of the kernel itself.  Direct access to some of these structures
is necessary for even the most primitive operations, while others reside at a
much lower level.


\paragraph{Message buffer}

The first structure we'll visit is the {\tt msgbuf} structure.  This particular
data structure can be thought of as a {\em template\/} for message data.  While
it is up to the programmer to define structures of this type, it is imperative
that you understand that there {\bf is} actually a structure of type {\tt msgbuf}.  It is declared in {\tt linux/msg.h} as follows: 



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
/* message buffer for msgsnd and msgrcv calls */
struct msgbuf {
    long mtype;         /* type of message */
    char mtext[1];      /* message text */
};
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




There are two members in the {\tt msgbuf} structure:

\begin{description}
\item[{\tt mtype}] \mbox{}

The message {\em type\/}, represented in a positive number.  This {\em must\/}
be a positive number!

\item[{\tt mtext}] \mbox{}

The message data itself.

\end{description}


The ability to assign a given message a {\em type\/}, essentially gives you the capability to
{\em multiplex\/} messages on a single queue.  For instance, client processes could be
assigned a magic number, which could be used as the message type for messages sent from a server
process.  The server itself could use some other number, which clients could use to send 
messages to it.  In another scenario, an application could mark error messages as having a message
type of 1, request messages could be type 2, etc.  The possibilities are endless.

On another note, do not be misled by the almost too-descriptive name assigned to the message
data element ({\tt mtext}).  This field is not restricted to holding only arrays of characters,
but any data, in any form.  The field itself is actually completely arbitrary, since this structure
gets redefined by the application programmer.  Consider this redefinition:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
struct my_msgbuf {
        long    mtype;          /* Message type */
        long    request_id;     /* Request identifier */
        struct  client info;    /* Client information structure */
};
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Here we see the message type, as before, but the remainder of the structure has been replaced
by two other elements, one of which is another structure!  This is the beauty of message queues.
The kernel makes no translations of data whatsoever.  Any information can be sent.  

There does exist an internal limit, however, of the maximum size of a given message.  In Linux,
this is defined in {\tt linux/msg.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
#define MSGMAX  4056   /* <= 4056 */   /* max size of message (bytes) */
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Messages can be no larger than 4,056 bytes in total size, including the {\tt mtype} member, 
which is 4 bytes in length ({\tt long}).




\paragraph{Kernel {\tt msg} structure}

The kernel stores each message in the queue within the framework of the {\tt msg} structure.
It is defined for us in {\tt linux/msg.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
/* one msg structure for each message */
struct msg {
    struct msg *msg_next;   /* next message on queue */
    long  msg_type;          
    char *msg_spot;         /* message text address */
    short msg_ts;           /* message text size */
};
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




\begin{description}
\item[{\tt msg\_next}] \mbox{}

This is a pointer to the next message in the queue.  They are stored as a
singly linked list within kernel addressing space.

\item[{\tt msg\_type}] \mbox{}

This is the message type, as assigned in the user structure {\tt msgbuf}.

\item[{\tt msg\_spot}] \mbox{}

A pointer to the beginning of the message body.

\item[{\tt msg\_ts}] \mbox{}

The length of the message text, or body.

\end{description}





\paragraph{Kernel {\tt msqid\_ds} structure}

Each of the three types of IPC objects has an internal data structure which is maintained
by the kernel.  For message queues, this is the {\tt msqid\_ds} structure.  The kernel creates,
stores, and maintains an instance of this structure for every message queue created on the
system.  It is defined in {\tt linux/msg.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
/* one msqid structure for each queue on the system */
struct msqid_ds {
    struct ipc_perm msg_perm;
    struct msg *msg_first;  /* first message on queue */
    struct msg *msg_last;   /* last message in queue */
    time_t msg_stime;       /* last msgsnd time */
    time_t msg_rtime;       /* last msgrcv time */
    time_t msg_ctime;       /* last change time */
    struct wait_queue *wwait;
    struct wait_queue *rwait;
    ushort msg_cbytes;    
    ushort msg_qnum;     
    ushort msg_qbytes;      /* max number of bytes on queue */
    ushort msg_lspid;       /* pid of last msgsnd */
    ushort msg_lrpid;       /* last receive pid */
};
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




While you will rarely have to concern yourself with most of the members of this structure, a brief
description of each is in order to complete our tour:

\begin{description}
\item[{\tt msg\_perm}] \mbox{}

An instance of the {\tt ipc\_perm} structure, which is defined
for us in {\tt linux/ipc.h}.  This holds the permission information for
the message queue, including the access permissions, and information
about the creator of the queue (uid, etc).

\item[{\tt msg\_first}] \mbox{}

Link to the first message in the queue (the head of the list).

\item[{\tt msg\_last}] \mbox{}

Link to the last message in the queue (the tail of the list).

\item[{\tt msg\_stime}] \mbox{}

Timestamp ({\tt time\_t}) of the last message that was sent to the queue.

\item[{\tt msg\_rtime}] \mbox{}

Timestamp of the last message retrieved from the queue.

\item[{\tt msg\_ctime}] \mbox{}

Timestamp of the last ``change'' made to the queue (more on this later).

\item[{\tt wwait}] \mbox{}

and

\item[{\tt rwait}] \mbox{}

Pointers into the kernel's {\em wait queue\/}.  They are
used when an operation on a message queue deems the process go into a sleep
state (i.e. queue is full and the process is waiting for an opening). 

\item[{\tt msg\_cbytes}] \mbox{}

Total number of bytes residing on the queue (sum of the sizes of all messages).

\item[{\tt msg\_qnum}] \mbox{}

Number of messages currently in the queue.

\item[{\tt msg\_qbytes}] \mbox{}

Maximum number of bytes on the queue.

\item[{\tt msg\_lspid}] \mbox{}

The PID of the process who sent the last message.

\item[{\tt msg\_lrpid}] \mbox{}

The PID of the process who retrieved the last message.

\end{description}





\paragraph{Kernel {\tt ipc\_perm} structure}

The kernel stores permission information for IPC objects in a structure of type {\tt ipc\_perm}.
For example, in the internal structure for a message queue described above, the {\tt msg\_perm}
member is of this type.  It is declared for us in {\tt linux/ipc.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

struct ipc_perm
{
  key_t  key;
  ushort uid;   /* owner euid and egid */
  ushort gid;
  ushort cuid;  /* creator euid and egid */
  ushort cgid;
  ushort mode;  /* access modes see mode flags below */
  ushort seq;   /* slot usage sequence number */
};
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




All of the above are fairly self-explanatory.  Stored along with the IPC key of the object is
information about both the creator and owner of the object (they may be different).  The octal
access modes are also stored here, as an {\tt unsigned short}.  Finally, the {\em slot usage sequence\/}
number is stored at the end.  Each time an IPC object is closed via a system call (destroyed), this
value gets incremented by the maximum number of IPC objects that can reside in a system.  Will you have to
concern yourself with this value? No.  

{\bf NOTE:}{\em  There is an excellent discussion on this topic, and the security reasons as to
its existence and behavior, in Richard Stevens' {\bf UNIX Network Programming} book, pp. 125.\/}




\subsubsection{SYSTEM CALL: msgget()}



In order to create a new message queue, or access an existing queue, the
{\tt msgget()} system call is used.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: msgget();                                                          

  PROTOTYPE: int msgget ( key_t key, int msgflg );                                             
    RETURNS: message queue identifier on success                                                       
             -1 on error: errno = EACCESS (permission denied)
                                  EEXIST (Queue exists, cannot create)
                                  EIDRM (Queue is marked for deletion)
                                  ENOENT (Queue does not exist)
                                  ENOMEM (Not enough memory to create queue)
                                  ENOSPC (Maximum queue limit exceeded)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The first argument to {\tt msgget()} is the key value (in our case returned
by a call to {\tt ftok()}).  This key value is then compared to existing key
values that exist within the kernel for other message queues.  At that point,  
the open or access operation is dependent upon the contents of the {\tt msgflg}
argument.

\begin{description}
\item[IPC\_CREAT] \mbox{}

Create the queue if it doesn't already exist in the kernel.

\item[IPC\_EXCL] \mbox{}

When used with IPC\_CREAT, fail if queue already exists.

\end{description}


If {\tt IPC\_CREAT} is used alone, {\tt msgget()} either returns the message queue
identifier for a newly created message queue, or returns the identifier for a queue
which exists with the same key value.  If {\tt IPC\_EXCL} is used along with {\tt IPC\_CREAT},
then either a new queue is created, or if the queue exists, the call fails with -1.
{\tt IPC\_EXCL} is useless by itself, but when combined with {\tt IPC\_CREAT}, it can
be used as a facility to guarantee that no existing queue is opened for access.

An optional octal mode may be OR'd into the mask, since each IPC object has
permissions that are similar in functionality to file permissions on a UNIX
file system!

Let's create a quick wrapper function for opening or creating message queue:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int open_queue( key_t keyval )
{
        int     qid;
        
        if((qid = msgget( keyval, IPC_CREAT | 0660 )) == -1)
        {
                return(-1);
        }
        
        return(qid);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Note the use of the explicit permissions of {\tt 0660}.  This small function either returns
a message queue identifier ({\tt int}), or -1 on error.  The key value must be passed to
it as its only argument.




\subsubsection{SYSTEM CALL: msgsnd()}



Once we have the queue identifier, we can begin performing operations on it.  To deliver a message
to a queue, you use the {\tt msgsnd} system call:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: msgsnd();                                                          

  PROTOTYPE: int msgsnd ( int msqid, struct msgbuf *msgp, int msgsz, int msgflg );
    RETURNS: 0 on success
             -1 on error: errno = EAGAIN (queue is full, and IPC_NOWAIT was asserted)
                                  EACCES (permission denied, no write permission)
                                  EFAULT (msgp address isn't accessable - invalid)
                                  EIDRM  (The message queue has been removed)
                                  EINTR  (Received a signal while waiting to write)
                                  EINVAL (Invalid message queue identifier, nonpositive
                                          message type, or invalid message size) 
                                  ENOMEM (Not enough memory to copy message buffer)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The first argument to {\tt msgsnd} is our queue identifier, returned by a previous
call to {\tt msgget}.  The second argument, {\tt msgp}, is a pointer to our
redeclared and loaded message buffer.  The {\tt msgsz} argument contains the size
of the message in bytes, excluding the length of the message type (4 byte long).

The {\tt msgflg} argument can be set to 0 (ignored), or:

\begin{description}
\item[IPC\_NOWAIT] \mbox{}

If the message queue is full, then the message is not written to the
queue, and control is returned to the calling process.  If not
specified, then the calling process will suspend (block) until the
message can be written.

\end{description}


Let's create another wrapper function for sending messages:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int send_message( int qid, struct mymsgbuf *qbuf )
{
        int     result, length;

        /* The length is essentially the size of the structure minus sizeof(mtype) */
        length = sizeof(struct mymsgbuf) - sizeof(long);        

        if((result = msgsnd( qid, qbuf, length, 0)) == -1)
        {
                return(-1);
        }
        
        return(result);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




This small function attempts to send the message residing at the passed address ({\tt qbuf}) 
to the message queue designated by the passed queue identifier ({\tt qid}).  Here is
a sample code snippet utilizing the two wrapper functions we have developed so far:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

#include <stdio.h>
#include <stdlib.h>
#include <linux/ipc.h>
#include <linux/msg.h>

main()
{
        int    qid;
        key_t  msgkey;
        struct mymsgbuf {
                long    mtype;          /* Message type */
                int     request;        /* Work request number */
                double  salary;         /* Employee's salary */
        } msg;

        /* Generate our IPC key value */
        msgkey = ftok(".", 'm');

        /* Open/create the queue */
        if(( qid = open_queue( msgkey)) == -1) {
                perror("open_queue");
                exit(1);
        }

        /* Load up the message with arbitrary test data */
        msg.mtype   = 1;        /* Message type must be a positive number! */   
        msg.request = 1;        /* Data element #1 */
        msg.salary  = 1000.00;  /* Data element #2 (my yearly salary!) */
        
        /* Bombs away! */
        if((send_message( qid, &msg )) == -1) {
                perror("send_message");
                exit(1);
        }
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




After creating/opening our message queue, we proceed to load up the message buffer
with test data ({\em note the lack of character data to illustrate our point about
sending binary information\/}).  A quick call to {\tt send\_message} merrily
distributes our message out to the message queue.

Now that we have a message on our queue, try the {\tt ipcs} command to view the status
of your queue.  Now let's turn the discussion to actually retrieving the message from
the queue.  To do this, you use the {\tt msgrcv()} system call:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: msgrcv();                                                          
  PROTOTYPE: int msgrcv ( int msqid, struct msgbuf *msgp, int msgsz, long mtype, int msgflg );
    RETURNS: Number of bytes copied into message buffer
             -1 on error: errno = E2BIG  (Message length is greater than msgsz, no MSG_NOERROR)
                                  EACCES (No read permission)
                                  EFAULT (Address pointed to by msgp is invalid)
                                  EIDRM  (Queue was removed during retrieval)
                                  EINTR  (Interrupted by arriving signal)
                                  EINVAL (msgqid invalid, or msgsz less than 0)
                                  ENOMSG (IPC_NOWAIT asserted, and no message exists
                                          in the queue to satisfy the request) 
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Obviously, the first argument is used to specify the queue to be used during the message
retrieval process (should have been returned by an earlier call to {\tt msgget}).  The
second argument ({\tt msgp}) represents the address of a message buffer variable to store
the retrieved message at.  The third argument ({\tt msgsz}) represents the size of the
message buffer structure, excluding the length of the {\tt mtype} member.  Once again,
this can easily be calculated as:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
msgsz = sizeof(struct mymsgbuf) - sizeof(long);
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}


The fourth argument ({\tt mtype}) specifies the {\em type\/} of message to retrieve from
the queue.  The kernel will search the queue for the oldest message having a matching type,
and will return a copy of it in the address pointed to by the {\tt msgp} argument.  One
special case exists.  If the {\tt mtype} argument is passed with a value of zero, then
the oldest message on the queue is returned, regardless of type.

If {\bf IPC\_NOWAIT} is passed as a flag, and no messages are available, the call
returns ENOMSG to the calling process.  Otherwise, the calling process blocks until a message
arrives in the queue that satisfies the {\tt msgrcv()} parameters.  If the queue is deleted
while a client is waiting on a message, EIDRM is returned.  EINTR is returned if a signal is
caught while the process is in the middle of blocking, and waiting for a message to arrive.

Let's examine a quick wrapper function for retrieving a message from our queue:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int read_message( int qid, long type, struct mymsgbuf *qbuf )
{
        int     result, length;

        /* The length is essentially the size of the structure minus sizeof(mtype) */
        length = sizeof(struct mymsgbuf) - sizeof(long);        

        if((result = msgrcv( qid, qbuf, length, type,  0)) == -1)
        {
                return(-1);
        }
        
        return(result);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




After successfully retrieving a message from the queue, the message entry within the queue
is destroyed.

The {\bf MSG\_NOERROR} bit in the {\tt msgflg} argument provides some additional capabilities.
If the size of the physical message data is greater than {\tt msgsz}, and {\bf MSG\_NOERROR}
is asserted, then the message is truncated, and only {\tt msgsz} bytes are returned.  Normally,
the {\tt msgrcv()} system call returns -1 ({\bf E2BIG}), and the message will remain on the
queue for later retrieval.  This behavior can used to create another wrapper function, which will allow
us to ``peek'' inside the queue, to see if a message has arrived that satisfies our request:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int peek_message( int qid, long type )
{
        int     result, length;

        if((result = msgrcv( qid, NULL, 0, type,  IPC_NOWAIT)) == -1)
        {
                if(errno == E2BIG)
                        return(TRUE);
        }
        
        return(FALSE);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Above, you will notice the lack of a buffer address and a length.  In this particular
case, we {\em want\/} the call to fail.  However, we check for the return of {\bf E2BIG} which
indicates that a message does exist which matches our requested type.  The wrapper function
returns {\bf TRUE} on success, {\bf FALSE} otherwise.  Also note the use of the {\bf IPC\_NOWAIT}
flag, which prevents the blocking behavior described earlier.




\subsubsection{SYSTEM CALL: msgctl()}

Through the development of the wrapper functions presented earlier, you now have a simple,
somewhat elegant approach to creating and utilizing message queues in your applications.  Now,
we will turn the discussion to directly manipulating the internal structures associated with
a given message queue.

To perform control operations on a message queue, you use the {\tt msgctl()} system call.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: msgctl();
  PROTOTYPE: int msgctl ( int msgqid, int cmd, struct msqid_ds *buf );
    RETURNS: 0 on success
             -1 on error: errno = EACCES (No read permission and cmd is IPC_STAT)
                                  EFAULT (Address pointed to by buf is invalid with IPC_SET and
                                          IPC_STAT commands)
                                  EIDRM  (Queue was removed during retrieval)
                                  EINVAL (msgqid invalid, or msgsz less than 0)
                                  EPERM  (IPC_SET or IPC_RMID command was issued, but
                                          calling process does not have write (alter)
                                          access to the queue)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Now, common sense dictates that direct manipulation of the internal kernel data structures could
lead to some late night fun.  Unfortunately, the resulting duties on the part of the programmer
could only be classified as fun if you like trashing the IPC subsystem.  By using {\tt msgctl()}
with a selective set of commands, you have the ability to manipulate those items which are less
likely to cause grief.  Let's look at these commands:

\begin{description}
\item[{\bf IPC\_STAT}] \mbox{}

Retrieves the msqid\_ds structure for a queue, and stores it in
the address of the buf argument.

\item[{\bf IPC\_SET}] \mbox{}

Sets the value of the ipc\_perm member of the msqid\_ds structure for
a queue.  Takes the values from the buf argument.

\item[{\bf IPC\_RMID}] \mbox{}

Removes the queue from the kernel.

\end{description}


Recall our discussion about the internal data structure for message queues ({\tt msqid\_ds}).
The kernel maintains an instance of this structure for each queue which exists in the system.
By using the {\bf IPC\_STAT} command, we can retrieve a copy of this structure for examination.
Let's look at a quick wrapper function that will retrieve the internal structure and copy it
into a passed address:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int get_queue_ds( int qid, struct msgqid_ds *qbuf )
{
        if( msgctl( qid, IPC_STAT, qbuf) == -1)
        {
                return(-1);
        }
        
        return(0);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




If we are unable to copy the internal buffer, -1 is returned to the calling function.  If all
went well, a value of 0 (zero) is returned, and the passed buffer should contain a copy of the
internal data structure for the message queue represented by the passed queue identifier ({\tt qid}).

Now that we have a copy of the internal data structure for a queue, what attributes can be 
manipulated, and how can we alter them?  The only modifiable item in the data structure is the
{\tt ipc\_perm} member.  This contains the permissions for the queue, as well as information
about the owner and creator.  However, the only members of the {\tt ipc\_perm} structure that
are modifiable are {\tt mode}, {\tt uid}, and {\tt gid}.  You can change the owner's
user id, the owner's group id, and the access permissions for the queue.

Let's create a wrapper function designed to change the mode of a queue.  The mode must be
passed in as a character array (i.e. {\em ``660''\/}).



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int change_queue_mode( int qid, char *mode )
{
        struct msqid_ds tmpbuf;

        /* Retrieve a current copy of the internal data structure */
        get_queue_ds( qid, &tmpbuf);

        /* Change the permissions using an old trick */
        sscanf(mode, "%ho", &tmpbuf.msg_perm.mode);

        /* Update the internal data structure */
        if( msgctl( qid, IPC_SET, &tmpbuf) == -1)
        {
                return(-1);
        }
        
        return(0);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




We retrieve a current copy of the internal data structure by a quick call to our {\tt get\_queue\_ds}
wrapper function.  We then make a call to {\tt sscanf()} to alter the {\tt mode} member
of the associated {\tt msg\_perm} structure.  No changes take place, however, until the new copy
is used to update the internal version.  This duty is performed by a call to {\tt msgctl()} using the
{\bf IPC\_SET} command.  

{\em BE CAREFUL!\/} It is possible to alter the permissions on a queue, and in doing so, inadvertently
lock yourself out!  Remember, these IPC objects don't go away unless they are properly removed, or
the system is rebooted.  So, even if you can't see a queue with {\tt ipcs} doesn't mean that it isn't
there.

\begin{quotation}
{\em To illustrate this point, a somewhat humorous anecdote seems to be in order.  While teaching a class on
UNIX internals at the University of South Florida, I ran into a rather embarrassing stumbling block.  I
had dialed into their lab server the night before, in order to compile and test the labwork to be used
in the week-long class.  In the process of my testing, I realized that I had made a typo in the
code used to alter the permissions on a message queue.  I created a simple message queue, and tested the 
sending and receiving capabilities with no incident.  However, when I attempted to change the mode of
the queue from ``660'' to ``600'', the resulting action was that I was locked out of my own queue!  As a 
result, I could not test the message queue labwork in the same area of my source directory.  Since I
used the ftok() function to create the IPC key, I was trying to access a queue that I did not have
proper permissions for.  I ended up contacting the local system administrator on the morning of the class,
only to spend an hour explaining to him what a message queue was, and why I needed him to run the ipcrm
command for me. grrrr.\/}
\end{quotation}


After successfully retrieving a message from a queue, the message is removed.  However, as 
mentioned earlier, IPC objects remain in the system unless explicitly removed, or the system
is rebooted.  Therefore, our message queue still exists within the kernel, available for use
long after a single message disappears.  To complete the life cycle of a message queue, they
should be removed with a call to {\tt msgctl()}, using the {\bf IPC\_RMID} command:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int remove_queue( int qid )
{
        if( msgctl( qid, IPC_RMID, 0) == -1)
        {
                return(-1);
        }
        
        return(0);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




This wrapper function returns 0 if the queue was removed without incident, else a value of -1.
The removal of the queue is atomic in nature, and any subsequent accesses to the queue for
whatever purpose will fail miserably.






\subsubsection{msgtool: An interactive message queue manipulator}

Few can deny the immediate benefit of having accurate technical information readily available.
Such materials provide a tremendous mechanism for learning and exploring new areas.  On the
same note, having real world examples to accompany any technical information will speed up
and reinforce the learning process.

Until now, the only useful examples which have been presented were the wrapper functions for
manipulating message queues.  While they are extremely useful, they have not been presented in
a manner which would warrant further study and experimentation.  To remedy this, you will be
presented with {\em msgtool\/}, an interactive command line utility for manipulating IPC message
queues.  While it certainly functions as an adequate tool for education reinforcement, it can
be applied directly into real world assignments, by providing message queue functionality via
standard shell scripts.




\paragraph{Background}

The {\tt msgtool} program relies on command line arguments to determine its behavior.  This is
what makes it especially useful when called from a shell script.  All of the capabilities are
provided, from creating, sending, and retrieving, to changing the permissions and finally
removing a queue.  Currently, it uses a character array for data, allowing you to send textual
messages.  Changing it to facilitate additional data types is left as an exercise to the reader.




\paragraph{Command Line Syntax}




\subparagraph{Sending Messages }

\begin{verbatim}
msgtool s (type) "text"
\end{verbatim}





\subparagraph{Retrieving Messages}

\begin{verbatim}
msgtool r (type)
\end{verbatim}





\subparagraph{Changing the Permissions (mode)}

\begin{verbatim}
msgtool m (mode)
\end{verbatim}





\subparagraph{Deleting a Queue}

\begin{verbatim}
msgtool d 
\end{verbatim}







\paragraph{Examples}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
msgtool  s   1 test
msgtool  s   5 test
msgtool  s   1 "This is a test"
msgtool  r   1
msgtool  d
msgtool  m   660
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}





\paragraph{The Source}

The following is the source code for the {\tt msgtool} facility.  It should compile clean
on any recent (decent) kernel revision that supports System V IPC.  Be sure to enable
System V IPC in your kernel when doing a rebuild!

On a side note, this utility will {\em create\/} a message queue if it does not exist, no 
matter what type of action is requested.

\begin{quotation}
{\bf NOTE:} {\em Since this tool uses the ftok() function to generate IPC key values, you may
encounter directory conflicts.  If you change directories at any point in your script, it probably 
won't work.  Another solution would be to hardcode a more complete path into msgtool, such as 
``/tmp/msgtool'', or possibly even allow the path to be passed on the command line, along with
the operational arguments. \/}
\end{quotation}




\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: msgtool.c
 *****************************************************************************
 A command line tool for tinkering with SysV style Message Queues
 *****************************************************************************/

#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>

#define MAX_SEND_SIZE 80

struct mymsgbuf {
        long mtype;
        char mtext[MAX_SEND_SIZE];
};

void send_message(int qid, struct mymsgbuf *qbuf, long type, char *text);
void read_message(int qid, struct mymsgbuf *qbuf, long type);
void remove_queue(int qid);
void change_queue_mode(int qid, char *mode);
void usage(void);


int main(int argc, char *argv[])
{
        key_t key;
        int   msgqueue_id;
        struct mymsgbuf qbuf;

        if(argc == 1)
                usage();

        /* Create unique key via call to ftok() */
        key = ftok(".", 'm');

        /* Open the queue - create if necessary */
        if((msgqueue_id = msgget(key, IPC_CREAT|0660)) == -1) {
                perror("msgget");
                exit(1);
        }
        
        switch(tolower(argv[1][0]))
        {
                case 's': send_message(msgqueue_id, (struct mymsgbuf *)&qbuf,
                                       atol(argv[2]), argv[3]); 
                          break;
                case 'r': read_message(msgqueue_id, &qbuf, atol(argv[2])); 
                          break;
                case 'd': remove_queue(msgqueue_id); 
                          break;        
                case 'm': change_queue_mode(msgqueue_id, argv[2]); 
                          break;

                 default: usage();

        }
        
        return(0);
}

void send_message(int qid, struct mymsgbuf *qbuf, long type, char *text)
{
        /* Send a message to the queue */
        printf("Sending a message ...\n");
        qbuf->mtype = type;
        strcpy(qbuf->mtext, text);

        if((msgsnd(qid, (struct msgbuf *)qbuf,
                strlen(qbuf->mtext)+1, 0)) ==-1)
        {
                perror("msgsnd");
                exit(1);
        }
}

void read_message(int qid, struct mymsgbuf *qbuf, long type)
{
        /* Read a message from the queue */
        printf("Reading a message ...\n");
        qbuf->mtype = type;
        msgrcv(qid, (struct msgbuf *)qbuf, MAX_SEND_SIZE, type, 0);
        
        printf("Type: %ld Text: %s\n", qbuf->mtype, qbuf->mtext);
}

void remove_queue(int qid)
{
        /* Remove the queue */
        msgctl(qid, IPC_RMID, 0);
}

void change_queue_mode(int qid, char *mode)
{
        struct msqid_ds myqueue_ds;

        /* Get current info */
        msgctl(qid, IPC_STAT, &myqueue_ds);

        /* Convert and load the mode */
        sscanf(mode, "%ho", &myqueue_ds.msg_perm.mode);

        /* Update the mode */
        msgctl(qid, IPC_SET, &myqueue_ds);
}

void usage(void)
{
        fprintf(stderr, "msgtool - A utility for tinkering with msg queues\n");
        fprintf(stderr, "\nUSAGE: msgtool (s)end <type> <messagetext>\n");
        fprintf(stderr, "               (r)ecv <type>\n");
        fprintf(stderr, "               (d)elete\n");
        fprintf(stderr, "               (m)ode <octal mode>\n");
        exit(1);
}

\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}





















\subsection{Semaphores}




\subsubsection{Basic Concepts}

Semaphores can best be described as counters used to control access to shared resources by multiple processes.
They are most often used as a locking mechanism to prevent processes from accessing a particular resource
while another process is performing operations on it.  Semaphores are often dubbed the most difficult to
grasp of the three types of System V IPC objects.  In order to fully understand semaphores, we'll discuss
them briefly before engaging any system calls and operational theory.

The name {\em semaphore\/} is actually an old railroad term, referring to the crossroad ``arms''
that prevent cars from crossing the tracks at intersections.  The same can be said about a simple
semaphore set.  If the semaphore is {\em on\/} (the arms are up), then a resource is available (cars may
cross the tracks).  However, if the semaphore is {\em off\/} (the arms are down), then resources are
not available (the cars must wait).

While this simple example may stand to introduce the concept, it is important to realize that
semaphores are actually implemented as {\em sets\/}, rather than as single entities.  Of course, a
given semaphore set might only have one semaphore, as in our railroad example.

Perhaps another approach to the concept of semaphores, is to think of them as {\em resource counters\/}.
Let's apply this concept to another real world scenario.  Consider a print spooler, capable of
handling multiple printers, with each printer handling multiple print requests.  A hypothetical print
spool manager will utilize semaphore sets to monitor access to each printer.

Assume that in our corporate print room, we have 5 printers online.  Our print spool manager allocates
a semaphore set with 5 semaphores in it, one for each printer on the system.  Since each printer is
only physically capable of printing one job at a time, each of our five semaphores in our set will
be initialized to a value of 1 (one), meaning that they are all online, and accepting requests.

John sends a print request to the spooler.  The print manager looks at the semaphore set, and finds
the first semaphore which has a value of one.  Before sending John's request to the physical device,
the print manager {\em decrements\/} the semaphore for the corresponding printer by a value of negative
one (-1).  Now, that semaphore's value is zero.  In the world of System V semaphores, a value of
zero represents 100\% resource utilization on that semaphore.  In our example, no other request can
be sent to that printer until it is no longer equal to zero.

When John's print job has completed, the print manager {\em increments\/} the value of the semaphore
which corresponds to the printer.  Its value is now back up to one (1), which means it is available again.
Naturally, if all 5 semaphores had a value of zero, that would indicate that they are all busy printing requests, and
that no printers are available.

Although this was a simple example, please do not be confused by the initial value of one (1) which
was assigned to each semaphore in the set.  Semaphores, when thought of as resource counters, may 
be initialized to {\em any positive\/} integer value, and are not limited to either being zero or one.  If it
were possible for each of our five printers to handle 10 print jobs at a time, we could initialize each
of our semaphores to 10, decrementing by one for every new job, and incrementing by one whenever a print 
job was finished.  As you will discover in the next chapter, semaphores have a close working relationship
with shared memory segments, acting as a {\em watchdog\/} to prevent multiple writes to the same memory
segment.  

Before delving into the associated system calls, lets take a brief tour through the various internal data
structures utilized during semaphore operations.




\subsubsection{Internal Data Structures}

Let's briefly look at data structures maintained by the kernel for semaphore sets.




\paragraph{Kernel {\tt semid\_ds} structure}

As with message queues, the kernel maintains a special internal data structure for each semaphore
set which exists within its addressing space.  This structure is of type {\tt semid\_ds}, and
is defined in {\tt linux/sem.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        /* One semid data structure for each set of semaphores in the system. */
        struct semid_ds {
                struct ipc_perm sem_perm;       /* permissions .. see ipc.h */
                time_t          sem_otime;      /* last semop time */
                time_t          sem_ctime;      /* last change time */
                struct sem      *sem_base;      /* ptr to first semaphore in array */
                struct wait_queue *eventn;
                struct wait_queue *eventz;
                struct sem_undo  *undo;         /* undo requests on this array */
                ushort          sem_nsems;      /* no. of semaphores in array */
        };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




As with message queues, operations on this structure are performed by a special system call, and
should not be tinkered with directly.  Here are descriptions of the more pertinent fields:

\begin{description}
\item[{\tt sem\_perm}] \mbox{}

This is an instance of the {\tt ipc\_perm} structure, which is defined
for us in {\tt linux/ipc.h}.  This holds the permission information for
the semaphore set, including the access permissions, and information
about the creator of the set (uid, etc).

\item[{\tt sem\_otime}] \mbox{}

Time of the last {\tt semop()} operation (more on this in a moment)

\item[{\tt sem\_ctime}] \mbox{}

Time of the last change to this structure (mode change, etc)

\item[{\tt sem\_base}] \mbox{}

Pointer to the first semaphore in the array (see next structure)

\item[{\tt sem\_undo}] \mbox{}

Number of {\em undo\/} requests in this array (more on this in a moment)

\item[{\tt sem\_nsems}] \mbox{}

Number of semaphores in the semaphore set (the array)

\end{description}





\paragraph{Kernel {\tt sem} structure}

In the {\tt semid\_ds} structure, there exists a pointer to the base of the semaphore array
itself.  Each array member is of the {\tt sem} structure type.  It is also defined in 
{\tt linux/sem.h}:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

        /* One semaphore structure for each semaphore in the system. */
        struct sem {
                short   sempid;         /* pid of last operation */
                ushort  semval;         /* current value */
                ushort  semncnt;        /* num procs awaiting increase in semval */
                ushort  semzcnt;        /* num procs awaiting semval = 0 */
        };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




\begin{description}
\item[{\tt sem\_pid}] \mbox{}

The PID (process ID) that performed the last operation

\item[{\tt sem\_semval}] \mbox{}

The current value of the semaphore

\item[{\tt sem\_semncnt}] \mbox{}

Number of processes waiting for resources to become available

\item[{\tt sem\_semzcnt}] \mbox{}

Number of processes waiting for 100\% resource utilization

\end{description}





\subsubsection{SYSTEM CALL: semget()}



In order to create a new semaphore set, or access an existing set, the
{\tt semget()} system call is used.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: semget();                                                          

  PROTOTYPE: int semget ( key_t key, int nsems, int semflg );                                             
    RETURNS: semaphore set IPC identifier on success
             -1 on error: errno = EACCESS (permission denied)
                                  EEXIST (set exists, cannot create (IPC_EXCL))
                                  EIDRM (set is marked for deletion)
                                  ENOENT (set does not exist, no IPC_CREAT was used)
                                  ENOMEM (Not enough memory to create new set)
                                  ENOSPC (Maximum set limit exceeded)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The first argument to {\tt semget()} is the key value (in our case returned
by a call to {\tt ftok()}).  This key value is then compared to existing key
values that exist within the kernel for other semaphore sets.  At that point,  
the open or access operation is dependent upon the contents of the {\tt semflg}
argument.

\begin{description}
\item[IPC\_CREAT] \mbox{}

Create the semaphore set if it doesn't already exist in the kernel.

\item[IPC\_EXCL] \mbox{}

When used with IPC\_CREAT, fail if semaphore set already exists.

\end{description}


If {\tt IPC\_CREAT} is used alone, {\tt semget()} either returns the semaphore set
identifier for a newly created set, or returns the identifier for a set
which exists with the same key value.  If {\tt IPC\_EXCL} is used along with {\tt IPC\_CREAT},
then either a new set is created, or if the set exists, the call fails with -1.
{\tt IPC\_EXCL} is useless by itself, but when combined with {\tt IPC\_CREAT}, it can
be used as a facility to guarantee that no existing semaphore set is opened for access.

As with the other forms of System V IPC, an optional octal mode may be OR'd
into the mask to form the permissions on the semaphore set.

The {\tt nsems} argument specifies the number of semaphores that should be created in
a new set.  This represents the number of printers in our fictional print room described
earlier.  The maximum number of semaphores in a set is defined in ``linux/sem.h'' as:

\begin{verbatim}
        #define SEMMSL  32      /* <=512 max num of semaphores per id */
\end{verbatim}




Note that the {\tt nsems} argument is ignored if you are explicitly opening an
existing set.

Let's create a wrapper function for opening or creating semaphore sets:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int open_semaphore_set( key_t keyval, int numsems )
{
        int     sid;

        if ( ! numsems )
                return(-1);
        
        if((sid = semget( mykey, numsems, IPC_CREAT | 0660 )) == -1)
        {
                return(-1);
        }
        
        return(sid);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Note the use of the explicit permissions of {\tt 0660}.  This small function either returns
a semaphore set identifier ({\tt int}), or -1 on error.  The key value must be passed to
it, as well as the number of semaphores to allocate space for if creating a new set.  In the example
presented at the end of this section, notice the use of the IPC\_EXCL flag to determine whether or
not the semaphore set exists or not.




\subsubsection{SYSTEM CALL: semop()}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: semop();                                                          
  PROTOTYPE: int semop ( int semid, struct sembuf *sops, unsigned nsops);
    RETURNS: 0 on success (all operations performed)
             -1 on error: errno = E2BIG (nsops greater than max number of ops allowed atomically)
                                  EACCESS (permission denied)
                                  EAGAIN (IPC_NOWAIT asserted, operation could not go through)
                                  EFAULT (invalid address pointed to by sops argument)
                                  EIDRM (semaphore set was removed)
                                  EINTR (Signal received while sleeping)
                                  EINVAL (set doesn't exist, or semid is invalid)
                                  ENOMEM (SEM_UNDO asserted, not enough memory to create the
                                          undo structure necessary)
                                  ERANGE (semaphore value out of range)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The first argument to {\tt semget()} is the key value (in our case returned by a call
to {\tt semget}).  The second argument ({\tt sops}) is a pointer to an array of {\em operations\/} to 
be performed on the semaphore set, while the third argument ({\tt nsops}) is the number
of operations in that array.

The {\tt sops} argument points to an array of type {\tt sembuf}.  This structure is
declared in {\tt linux/sem.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        /* semop system call takes an array of these */
        struct sembuf {
                ushort  sem_num;        /* semaphore index in array */
                short   sem_op;         /* semaphore operation */
                short   sem_flg;        /* operation flags */
        };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




\begin{description}
\item[{\tt sem\_num}] \mbox{}

The number of the semaphore you wish to deal with

\item[{\tt sem\_op}] \mbox{}

The operation to perform (positive, negative, or zero)

\item[{\tt sem\_flg}] \mbox{}

Operational flags

\end{description}


If {\tt sem\_op} is negative, then its value is subtracted from the semaphore. 
This correlates with obtaining resources that the semaphore controls or monitors access of.
If {\bf IPC\_NOWAIT} is not specified, then the calling process sleeps until the requested amount of
resources are available in the semaphore (another process has released some).

If {\tt sem\_op} is positive, then it's value is added to the semaphore.
This correlates with returning resources back to the application's semaphore set.
Resources should always be returned to a semaphore set when they are no longer needed!

Finally, if {\tt sem\_op} is zero (0), then the calling process will sleep() until the
semaphore's value is 0.  This correlates to waiting for a semaphore to reach 100\% utilization.
A good example of this would be a daemon running with superuser permissions that could
dynamically adjust the size of the semaphore set if it reaches full utilization. 

In order to explain the {\tt semop} call, let's revisit our print room scenario.  Let's
assume only one printer, capable of only one job at a time.  We create a semaphore set with
only one semaphore in it (only one printer), and initialize that one semaphore to a value of one (only
one job at a time).

Each time we desire to send a job to this printer, we need to first make sure that the resource
is available.  We do this by attempting to obtain one {\em unit\/} from the semaphore.  Let's
load up a sembuf array to perform the operation:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        struct sembuf sem_lock = { 0, -1, IPC_NOWAIT };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Translation of the above initialized structure dictates that a value of ``-1'' will be added
to semaphore number 0 in the semaphore set.  In other words, one unit of resources will be
obtained from the only semaphore in our set (0th member).  {\bf IPC\_NOWAIT} is specified, 
so the call will either go through immediately, or fail if another print job is currently
printing.  Here is an example of using this initialized {\tt sembuf} structure with the {\tt semop}
system call:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        if((semop(sid, &sem_lock, 1) == -1)
                perror("semop");
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The third argument ({\tt nsops}) says that we are only performing one (1) operation (there
is only one {\tt sembuf} structure in our array of operations).  The {\tt sid} argument
is the IPC identifier for our semaphore set.

When our print job has completed, we must {\em return\/} the resources back to the semaphore
set, so that others may use the printer.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        struct sembuf sem_unlock = { 0, 1, IPC_NOWAIT };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Translation of the above initialized structure dictates that a value of ``1'' will be added to
semaphore number 0 in the semaphore set.  In other words, one unit of resources will be returned
to the set.






\subsubsection{SYSTEM CALL: semctl()}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: semctl();                                                          
  PROTOTYPE: int semctl ( int semid, int semnum, int cmd, union semun arg );
    RETURNS: positive integer on success 
             -1 on error: errno = EACCESS (permission denied)
                                  EFAULT (invalid address pointed to by arg argument)
                                  EIDRM (semaphore set was removed)
                                  EINVAL (set doesn't exist, or semid is invalid)
                                  EPERM (EUID has no privileges for cmd in arg)
                                  ERANGE (semaphore value out of range)
  NOTES: Performs control operations on a semaphore set
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The {\em semctl\/} system call is used to perform control operations on a semaphore set.  This
call is analogous to the {\em msgctl\/} system call which is used for operations on message
queues.  If you compare the argument lists of the two system calls, you will notice that the list
for {\em semctl\/} varies slightly from that of {\em msgctl\/}.  Recall that semaphores are
actually implemented as sets, rather than as single entities.  With semaphore operations, not only 
does the IPC key need to be passed, but the target semaphore within the set as well.  

Both system calls utilize a {\em cmd\/} argument, for specification of the command to be performed
on the IPC object.  The remaining difference lies in the final argument to both calls.  In {\em msgctl\/},
the final argument represents a copy of the internal data structure used by the kernel.  Recall that we
used this structure to retrieve internal information about a message queue, as well as to set or change
permissions and ownership of the queue.  With semaphores, additional operational commands are supported,
thus requiring a more complex data type as the final argument.  The use of a {\em union\/} confuses many
neophyte semaphore programmers to a substantial degree.  We will dissect this structure carefully, in
an effort to prevent any confusion.

The first argument to {\tt semctl()} is the key value (in our case returned by a call
to {\tt semget}).  The second argument ({\tt semun}) is the semaphore number that 
an operation is targeted towards.  In essence, this can be thought of as an {\em index\/} into 
the semaphore set, with the first semaphore (or only one) in the set being represented by a 
value of zero (0).

The {\tt cmd} argument represents the command to be performed against the set.  As you can see,
the familiar IPC\_STAT/IPC\_SET commands are present, along with a wealth of additional commands
specific to semaphore sets:

\begin{description}
\item[{\bf IPC\_STAT}] \mbox{}

Retrieves the semid\_ds structure for a set, and stores it in
the address of the buf argument in the semun union.

\item[{\bf IPC\_SET}] \mbox{}

Sets the value of the ipc\_perm member of the semid\_ds structure for
a set.  Takes the values from the buf argument of the semun union.

\item[{\bf IPC\_RMID}] \mbox{}

Removes the set from the kernel.

\item[{\bf GETALL}] \mbox{}

Used to obtain the values of all semaphores in a set.  The integer values are
stored in an array of unsigned short integers pointed to by the {\em array\/} member of the union.

\item[{\bf GETNCNT}] \mbox{}

Returns the number of processes currently waiting for resources.

\item[{\bf GETPID}] \mbox{}

Returns the PID of the process which performed the last {\em semop\/} call.

\item[{\bf GETVAL}] \mbox{}

Returns the value of a single semaphore within the set.

\item[{\bf GETZCNT}] \mbox{}

Returns the number of processes currently waiting for 100\% resource utilization.

\item[{\bf SETALL}] \mbox{}

Sets all semaphore values with a set to the matching values contained in the
{\em array\/} member of the union.

\item[{\bf SETVAL}] \mbox{}

Sets the value of an individual semaphore within the set to the {\em val\/} member of
the union.

\end{description}


The {\tt arg} argument represents an instance of type {\tt semun}.  This particular union is
declared in {\tt linux/sem.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        /* arg for semctl system calls. */
        union semun {
                int val;                /* value for SETVAL */
                struct semid_ds *buf;   /* buffer for IPC_STAT & IPC_SET */
                ushort *array;          /* array for GETALL & SETALL */
                struct seminfo *__buf;  /* buffer for IPC_INFO */
                void *__pad;
        };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




\begin{description}
\item[{\tt val}] \mbox{}

Used when the SETVAL command is performed.  Specifies the value to set the semaphore to.

\item[{\tt buf}] \mbox{}

Used in the IPC\_STAT/IPC\_SET commands.  Represents a copy of the internal semaphore data
structure used in the kernel.

\item[{\tt array}] \mbox{}

A pointer used in the GETALL/SETALL commands.  Should point to an array of integer values
to be used in setting or retrieving all semaphore values in a set.

\end{description}


The remaining arguments {\em \_\_buf\/} and {\em \_\_pad\/} are used internally in the semaphore code within
the kernel, and are of little or no use to the application developer.  As a matter of fact, these two
arguments are specific to the Linux operating system, and are not found in other UNIX implementations.

Since this particular system call is arguably the most difficult to grasp of all the System V IPC calls,
we'll examine multiple examples of it in action.

The following snippet returns the value of the passed semaphore.  The final argument (the union) is
ignored when the GETVAL command is used:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int get_sem_val( int sid, int semnum )
{
        return( semctl(sid, semnum, GETVAL, 0));
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




To revisit the printer example, let's say the status of all five printers was required:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

        #define MAX_PRINTERS 5

        printer_usage()
        {
                int x;

                for(x=0; x<MAX_PRINTERS; x++)
                        printf("Printer %d: %d\n\r", x, get_sem_val( sid, x ));
        }
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Consider the following function, which could be used to initialize a new semaphore value:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
void init_semaphore( int sid, int semnum, int initval)
{
        union semun semopts;    

        semopts.val = initval;
        semctl( sid, semnum, SETVAL, semopts);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Note that the final argument of {\em semctl\/} is a copy of the union, rather than a pointer to it.
While we're on the subject of the union as an argument, allow me to demonstrate a rather common mistake
when using this system call.

Recall from the msgtool project that the IPC\_STAT and IPC\_SET commands were used to alter permissions
on the queue.  While these commands are supported in the semaphore implementation, their usage is a
bit different, as the internal data structure is retrieved and copied from a member of the union, rather
than as a single entity.  Can you locate the bug in this code?



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

/* Required permissions should be passed in as text (ex: "660") */

void changemode(int sid, char *mode)
{
        int rc;
        struct semid_ds mysemds;

        /* Get current values for internal data structure */
        if((rc = semctl(sid, 0, IPC_STAT, semopts)) == -1)
        {
                perror("semctl");
                exit(1);
        }
                
        printf("Old permissions were %o\n", semopts.buf->sem_perm.mode);
                
        /* Change the permissions on the semaphore */
        sscanf(mode, "%o", &semopts.buf->sem_perm.mode);

        /* Update the internal data structure */
        semctl(sid, 0, IPC_SET, semopts);

        printf("Updated...\n");
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




The code is attempting to make a local copy of the internal data structure for the set, modify the
permissions, and IPC\_SET them back to the kernel.  However, the first call to {\em semctl\/} promptly
returns EFAULT, or bad address for the last argument (the union!).  In addition, if we hadn't checked for
errors from that call, we would have gotten a memory fault.  Why?

Recall that the IPC\_SET/IPC\_STAT commands use the {\em buf\/} member of the union, which is a {\em pointer\/}
to a type {\em semid\_ds\/}.  Pointers are pointers are pointers are pointers!  The {\em buf\/} member must
point to some valid storage location in order for our function to work properly.  Consider this revamped
version:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

void changemode(int sid, char *mode)
{
        int rc;
        struct semid_ds mysemds;

        /* Get current values for internal data structure */

        /* Point to our local copy first! */
        semopts.buf = &mysemds;

        /* Let's try this again! */
        if((rc = semctl(sid, 0, IPC_STAT, semopts)) == -1)
        {
                perror("semctl");
                exit(1);
        }
                
        printf("Old permissions were %o\n", semopts.buf->sem_perm.mode);
                
        /* Change the permissions on the semaphore */
        sscanf(mode, "%o", &semopts.buf->sem_perm.mode);

        /* Update the internal data structure */
        semctl(sid, 0, IPC_SET, semopts);

        printf("Updated...\n");
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}







\subsubsection{semtool: An interactive semaphore manipulator}






\paragraph{Background}

The {\tt semtool} program relies on command line arguments to determine its behavior.  This is
what makes it especially useful when called from a shell script.  All of the capabilities are
provided, from creating and manipulating, to changing the permissions and finally
removing a semaphore set.  It can be used to control shared resources via standard shell scripts.




\paragraph{Command Line Syntax}




\subparagraph{Creating a Semaphore Set}

\begin{verbatim}
semtool c (number of semaphores in set)
\end{verbatim}





\subparagraph{Locking a Semaphore}

\begin{verbatim}
semtool l (semaphore number to lock)
\end{verbatim}





\subparagraph{Unlocking a Semaphore}

\begin{verbatim}
semtool u (semaphore number to unlock)
\end{verbatim}





\subparagraph{Changing the Permissions (mode)}

\begin{verbatim}
semtool m (mode)
\end{verbatim}





\subparagraph{Deleting a Semaphore Set}

\begin{verbatim}
semtool d 
\end{verbatim}







\paragraph{Examples}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
semtool  c 5
semtool  l
semtool  u
semtool  m 660
semtool  d
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}





\paragraph{The Source}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: semtool.c
 *****************************************************************************
 A command line tool for tinkering with SysV style Semaphore Sets

 *****************************************************************************/

#include <stdio.h>
#include <ctype.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>

#define SEM_RESOURCE_MAX        1       /* Initial value of all semaphores */

void opensem(int *sid, key_t key);
void createsem(int *sid, key_t key, int members);
void locksem(int sid, int member);
void unlocksem(int sid, int member);
void removesem(int sid);
unsigned short get_member_count(int sid);
int getval(int sid, int member);
void dispval(int sid, int member);
void changemode(int sid, char *mode);
void usage(void);

int main(int argc, char *argv[])
{
        key_t key;
        int   semset_id;

        if(argc == 1)
                usage();

        /* Create unique key via call to ftok() */
        key = ftok(".", 's');

        switch(tolower(argv[1][0]))
        {
                case 'c': if(argc != 3)
                                usage();
                          createsem(&semset_id, key,  atoi(argv[2]));
                          break;
                case 'l': if(argc != 3)
                                usage();
                          opensem(&semset_id, key);
                          locksem(semset_id, atoi(argv[2]));
                          break;
                case 'u': if(argc != 3)
                                usage();
                          opensem(&semset_id, key);
                          unlocksem(semset_id, atoi(argv[2]));
                          break;
                case 'd': opensem(&semset_id, key);
                          removesem(semset_id);
                          break;        
                case 'm': opensem(&semset_id, key);
                          changemode(semset_id, argv[2]);
                          break;        
                 default: usage();

        }
        
        return(0);
}

void opensem(int *sid, key_t key)
{
        /* Open the semaphore set - do not create! */

        if((*sid = semget(key, 0, 0666)) == -1) 
        {
                printf("Semaphore set does not exist!\n");
                exit(1);
        }

}

void createsem(int *sid, key_t key, int members)
{
        int cntr;
        union semun semopts;

        if(members > SEMMSL) {
                printf("Sorry, max number of semaphores in a set is %d\n",
                        SEMMSL);
                exit(1);
        }

        printf("Attempting to create new semaphore set with %d members\n",
                                members);

        if((*sid = semget(key, members, IPC_CREAT|IPC_EXCL|0666))
                        == -1) 
        {
                fprintf(stderr, "Semaphore set already exists!\n");
                exit(1);
        }

        semopts.val = SEM_RESOURCE_MAX;
        
        /* Initialize all members (could be done with SETALL) */        
        for(cntr=0; cntr<members; cntr++)
                semctl(*sid, cntr, SETVAL, semopts);
}

void locksem(int sid, int member)
{
        struct sembuf sem_lock={ 0, -1, IPC_NOWAIT};

        if( member<0 || member>(get_member_count(sid)-1))
        {
                fprintf(stderr, "semaphore member %d out of range\n", member);
                return;
        }

        /* Attempt to lock the semaphore set */
        if(!getval(sid, member))
        {
                fprintf(stderr, "Semaphore resources exhausted (no lock)!\n");
                exit(1);
        }
        
        sem_lock.sem_num = member;
        
        if((semop(sid, &sem_lock, 1)) == -1)
        {
                fprintf(stderr, "Lock failed\n");
                exit(1);
        }
        else
                printf("Semaphore resources decremented by one (locked)\n");

        dispval(sid, member);
}

void unlocksem(int sid, int member)
{
        struct sembuf sem_unlock={ member, 1, IPC_NOWAIT};
        int semval;

        if( member<0 || member>(get_member_count(sid)-1))
        {
                fprintf(stderr, "semaphore member %d out of range\n", member);
                return;
        }

        /* Is the semaphore set locked? */
        semval = getval(sid, member);
        if(semval == SEM_RESOURCE_MAX) {
                fprintf(stderr, "Semaphore not locked!\n");
                exit(1);
        }

        sem_unlock.sem_num = member;

        /* Attempt to lock the semaphore set */
        if((semop(sid, &sem_unlock, 1)) == -1)
        {
                fprintf(stderr, "Unlock failed\n");
                exit(1);
        }
        else
                printf("Semaphore resources incremented by one (unlocked)\n");

        dispval(sid, member);
}

void removesem(int sid)
{
        semctl(sid, 0, IPC_RMID, 0);
        printf("Semaphore removed\n");
}

unsigned short get_member_count(int sid)
{
        union semun semopts;
        struct semid_ds mysemds;

        semopts.buf = &mysemds;

        /* Return number of members in the semaphore set */
        return(semopts.buf->sem_nsems);
}

int getval(int sid, int member)
{
        int semval;

        semval = semctl(sid, member, GETVAL, 0);
        return(semval);
}

void changemode(int sid, char *mode)
{
        int rc;
        union semun semopts;
        struct semid_ds mysemds;

        /* Get current values for internal data structure */
        semopts.buf = &mysemds;

        rc = semctl(sid, 0, IPC_STAT, semopts);

        if (rc == -1) {
                perror("semctl");
                exit(1);
        }
                
        printf("Old permissions were %o\n", semopts.buf->sem_perm.mode);
                
        /* Change the permissions on the semaphore */
        sscanf(mode, "%ho", &semopts.buf->sem_perm.mode);

        /* Update the internal data structure */
        semctl(sid, 0, IPC_SET, semopts);

        printf("Updated...\n");

}

void dispval(int sid, int member)
{
        int semval;

        semval = semctl(sid, member, GETVAL, 0);
        printf("semval for member %d is %d\n", member, semval);
}

void usage(void)
{
        fprintf(stderr, "semtool - A utility for tinkering with semaphores\n");
        fprintf(stderr, "\nUSAGE:  semtool4 (c)reate <semcount>\n");
        fprintf(stderr, "                 (l)ock <sem #>\n");
        fprintf(stderr, "                 (u)nlock <sem #>\n");
        fprintf(stderr, "                 (d)elete\n");
        fprintf(stderr, "                 (m)ode <mode>\n");
        exit(1);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}







\subsubsection{semstat: A semtool companion program}

As an added bonus, the source code to a companion program called {\tt semstat}
is provided next.  The {\tt semstat} program displays the values of each of the
semaphores in the set created by {\tt semtool}.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

/*****************************************************************************
 Excerpt from "Linux Programmer's Guide - Chapter 6"
 (C)opyright 1994-1995, Scott Burkett
 ***************************************************************************** 
 MODULE: semstat.c
 *****************************************************************************
 A companion command line tool for the semtool package.  semstat displays
 the current value of all semaphores in the set created by semtool.
 *****************************************************************************/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>

int get_sem_count(int sid);
void show_sem_usage(int sid);
int get_sem_count(int sid);
void dispval(int sid);

int main(int argc, char *argv[])
{
        key_t key;
        int   semset_id;

        /* Create unique key via call to ftok() */
        key = ftok(".", 's');

        /* Open the semaphore set - do not create! */
        if((semset_id = semget(key, 1, 0666)) == -1) 
        {
                printf("Semaphore set does not exist\n");
                exit(1);
        }

        show_sem_usage(semset_id); 
        return(0);
}

void show_sem_usage(int sid)
{
        int cntr=0, maxsems, semval;

        maxsems = get_sem_count(sid);

        while(cntr < maxsems) {
                semval = semctl(sid, cntr, GETVAL, 0);
                printf("Semaphore #%d:  --> %d\n", cntr, semval);
                cntr++;
        }
}

int get_sem_count(int sid)
{
        int rc;
        struct semid_ds mysemds;
        union semun semopts;

        /* Get current values for internal data structure */
        semopts.buf = &mysemds;

        if((rc = semctl(sid, 0, IPC_STAT, semopts)) == -1) {
                perror("semctl");
                exit(1);
        }

        /* return number of semaphores in set */
        return(semopts.buf->sem_nsems);
}

void dispval(int sid)
{
        int semval;

        semval = semctl(sid, 0, GETVAL, 0);
        printf("semval is %d\n", semval);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}















\subsection{Shared Memory}






\subsubsection{Basic Concepts}

Shared memory can best be described as the mapping of an area (segment) of memory
that will be mapped and shared by more than one process.  This is by far the fastest
form of IPC, because there is no intermediation (i.e. a pipe, a message queue, etc).
Instead, information is mapped directly from a memory segment, and into the addressing space
of the calling process.  A segment can be created by one process, and subsequently written to and
read from by any number of processes.




\subsubsection{Internal and User Data Structures}

Let's briefly look at data structures maintained by the kernel for shared memory segments.


\paragraph{Kernel {\tt shmid\_ds} structure}

As with message queues and semaphore sets, the kernel maintains a special internal data structure for
each shared memory segment which exists within its addressing space.  This structure is of type
{\tt shmid\_ds}, and is defined in {\tt linux/shm.h} as follows:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
        /* One shmid data structure for each shared memory segment in the system. */
        struct shmid_ds {
                struct ipc_perm shm_perm;        /* operation perms */
                int     shm_segsz;               /* size of segment (bytes) */
                time_t  shm_atime;               /* last attach time */
                time_t  shm_dtime;               /* last detach time */
                time_t  shm_ctime;               /* last change time */
                unsigned short  shm_cpid;        /* pid of creator */
                unsigned short  shm_lpid;        /* pid of last operator */
                short   shm_nattch;              /* no. of current attaches */

                                                 /* the following are private */

                unsigned short   shm_npages;     /* size of segment (pages) */
                unsigned long   *shm_pages;      /* array of ptrs to frames -> SHMMAX */ 
                struct vm_area_struct *attaches; /* descriptors for attaches */
        };
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Operations on this structure are performed by a special system call, and
should not be tinkered with directly.  Here are descriptions of the more pertinent fields:

\begin{description}
\item[{\tt shm\_perm}] \mbox{}

This is an instance of the {\tt ipc\_perm} structure, which is defined
for us in {\tt linux/ipc.h}.  This holds the permission information for
the segment, including the access permissions, and information
about the creator of the segment (uid, etc).

\item[{\tt shm\_segsz}] \mbox{}

Size of the segment (measured in bytes).

\item[{\tt shm\_atime}] \mbox{}

Time the last process attached the segment.

\item[{\tt shm\_dtime}] \mbox{}

Time the last process detached the segment.

\item[{\tt shm\_ctime}] \mbox{}

Time of the last change to this structure (mode change, etc).

\item[{\tt shm\_cpid}] \mbox{}

The PID of the creating process.

\item[{\tt shm\_lpid}] \mbox{}

The PID of the last process to operate on the segment.

\item[{\tt shm\_nattch}] \mbox{}

Number of processes currently attached to the segment.

\end{description}





\subsubsection{SYSTEM CALL: shmget()}

In order to create a new message queue, or access an existing queue, the
{\tt shmget()} system call is used.



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: shmget();                                                          

  PROTOTYPE: int shmget ( key_t key, int size, int shmflg );                                             
    RETURNS: shared memory segment identifier on success                                                       
             -1 on error: errno = EINVAL (Invalid segment size specified)
                                  EEXIST (Segment exists, cannot create)
                                  EIDRM (Segment is marked for deletion, or was removed)
                                  ENOENT (Segment does not exist)
                                  EACCES (Permission denied)
                                  ENOMEM (Not enough memory to create segment)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




This particular call should almost seem like old news at this point.  It is strikingly similar to
the corresponding {\tt get} calls for message queues and semaphore sets.

The first argument to {\tt shmget()} is the key value (in our case returned
by a call to {\tt ftok()}).  This key value is then compared to existing key
values that exist within the kernel for other shared memory segments.  At that point,  
the open or access operation is dependent upon the contents of the {\tt shmflg}
argument.

\begin{description}
\item[IPC\_CREAT] \mbox{}

Create the segment if it doesn't already exist in the kernel.

\item[IPC\_EXCL] \mbox{}

When used with IPC\_CREAT, fail if segment already exists.

\end{description}


If {\tt IPC\_CREAT} is used alone, {\tt shmget()} either returns the segment
identifier for a newly created segment, or returns the identifier for a segment
which exists with the same key value.  If {\tt IPC\_EXCL} is used along with {\tt IPC\_CREAT},
then either a new segment is created, or if the segment exists, the call fails with -1.
{\tt IPC\_EXCL} is useless by itself, but when combined with {\tt IPC\_CREAT}, it can
be used as a facility to guarantee that no existing segment is opened for access.

Once again, an optional octal mode may be OR'd into the mask.

Let's create a wrapper function for locating or creating a shared memory segment :



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
int open_segment( key_t keyval, int segsize )
{
        int     shmid;
        
        if((shmid = shmget( keyval, segsize, IPC_CREAT | 0660 )) == -1)
        {
                return(-1);
        }
        
        return(shmid);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Note the use of the explicit permissions of {\tt 0660}.  This small function either returns
a shared memory segment identifier ({\tt int}), or -1 on error.  The key value and requested
segment size (in bytes) are passed as arguments.

Once a process has a valid IPC identifier for a given segment, the next step is for the process to
{\tt attach} or map the segment into its own addressing space.




\subsubsection{SYSTEM CALL: shmat()}



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: shmat();                                                          

  PROTOTYPE: int shmat ( int shmid, char *shmaddr, int shmflg);
    RETURNS: address at which segment was attached to the process, or
             -1 on error: errno = EINVAL (Invalid IPC ID value or attach address passed)
                                  ENOMEM (Not enough memory to attach segment)
                                  EACCES (Permission denied)
  NOTES: 
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




If the addr argument is zero (0), the kernel tries to find an unmapped region.  This is the recommended
method.  An address can be specified, but is typically only used to facilitate proprietary hardware or
to resolve conflicts with other apps.  The SHM\_RND flag can be OR'd into the flag argument to force
a passed address to be page aligned (rounds down to the nearest page size).

In addition, if the SHM\_RDONLY flag is OR'd in with the flag argument, then the shared memory segment
will be mapped in, but marked as readonly.

This call is perhaps the simplest to use.  Consider this wrapper function, which is passed a valid
IPC identifier for a segment, and returns the address that the segment was attached to:



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}

char *attach_segment( int shmid )
{
        return(shmat(shmid, 0, 0));
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




Once a segment has been properly attached, and a process has a pointer to the start of that
segment, reading and writing to the segment become as easy as simply referencing or dereferencing
the pointer!  Be careful not to lose the value of the original pointer!  If this happens, you will
have no way of accessing the base (start) of the segment.




\subsubsection{SYSTEM CALL: shmctl()}



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: shmctl();
  PROTOTYPE: int shmctl ( int shmqid, int cmd, struct shmid_ds *buf );
    RETURNS: 0 on success
             -1 on error: errno = EACCES (No read permission and cmd is IPC_STAT)
                                  EFAULT (Address pointed to by buf is invalid with IPC_SET and
                                          IPC_STAT commands)
                                  EIDRM  (Segment was removed during retrieval)
                                  EINVAL (shmqid invalid)
                                  EPERM  (IPC_SET or IPC_RMID command was issued, but
                                          calling process does not have write (alter)
                                          access to the segment)
      NOTES:
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




This particular call is modeled directly after the {\em msgctl\/} call for message queues.  In light of
this fact, it won't be discussed in too much detail.  Valid command values are:

\begin{description}
\item[{\bf IPC\_STAT}] \mbox{}

Retrieves the shmid\_ds structure for a segment, and stores it in
the address of the buf argument

\item[{\bf IPC\_SET}] \mbox{}

Sets the value of the ipc\_perm member of the shmid\_ds structure for
a segment.  Takes the values from the buf argument.

\item[{\bf IPC\_RMID}] \mbox{}

Marks a segment for removal.

\end{description}


The IPC\_RMID command doesn't actually remove a segment from the kernel.  Rather, it {\tt marks}
the segment for removal.  The actual removal itself occurs when the last process currently attached
to the segment has properly detached it.  Of course, if no processes are currently attached to the
segment, the removal seems immediate.

To properly detach a shared memory segment, a process calls the {\em shmdt\/} system call.




\subsubsection{SYSTEM CALL: shmdt()}



\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
  SYSTEM CALL: shmdt();                                                          

  PROTOTYPE: int shmdt ( char *shmaddr );
    RETURNS: -1 on error: errno = EINVAL (Invalid attach address passed)
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




After a shared memory segment is no longer needed by a process, it should be detached by calling
this system call.  As mentioned earlier, this is not the same as removing the segment from the kernel!
After a detach is successful, the shm\_nattch member of the associates shmid\_ds structure is decremented
by one.  When this value reaches zero (0), the kernel will physically remove the segment.




\subsubsection{shmtool: An interactive shared memory manipulator}






\paragraph{Background}

Our final example of System V IPC objects will be {\tt shmtool}, which is a command line tool
for creating, reading, writing, and deleting shared memory segments.  Once again, like the previous
examples, the segment is created during any operation, if it did not previously exist.




\paragraph{Command Line Syntax}




\subparagraph{Writing strings to the segment}

\begin{verbatim}
shmtool w "text"
\end{verbatim}





\subparagraph{Retrieving strings from the segment}

\begin{verbatim}
shmtool r 
\end{verbatim}





\subparagraph{Changing the Permissions (mode)}

\begin{verbatim}
shmtool m (mode)
\end{verbatim}





\subparagraph{Deleting the segment}

\begin{verbatim}
shmtool d 
\end{verbatim}







\paragraph{Examples}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
shmtool  w   test
shmtool  w   "This is a test"
shmtool  r   
shmtool  d
shmtool  m   660
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}





\paragraph{The Source}

\par
\addvspace{\medskipamount}
\nopagebreak\hrule
\begin{verbatim}
#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>

#define SEGSIZE 100

main(int argc, char *argv[])
{
        key_t key;
        int   shmid, cntr;
        char  *segptr;

        if(argc == 1)
                usage();

        /* Create unique key via call to ftok() */
        key = ftok(".", 'S');

        /* Open the shared memory segment - create if necessary */
        if((shmid = shmget(key, SEGSIZE, IPC_CREAT|IPC_EXCL|0666)) == -1) 
        {
                printf("Shared memory segment exists - opening as client\n");

                /* Segment probably already exists - try as a client */
                if((shmid = shmget(key, SEGSIZE, 0)) == -1) 
                {
                        perror("shmget");
                        exit(1);
                }
        }
        else
        {
                printf("Creating new shared memory segment\n");
        }

        /* Attach (map) the shared memory segment into the current process */
        if((segptr = shmat(shmid, 0, 0)) == -1)
        {
                perror("shmat");
                exit(1);
        }
        
        switch(tolower(argv[1][0]))
        {
                case 'w': writeshm(shmid, segptr, argv[2]);
                          break;
                case 'r': readshm(shmid, segptr);
                          break;
                case 'd': removeshm(shmid);
                          break;
                case 'm': changemode(shmid, argv[2]);
                          break;
                 default: usage();

        }
}

writeshm(int shmid, char *segptr, char *text)
{
        strcpy(segptr, text);
        printf("Done...\n");
}

readshm(int shmid, char *segptr)
{
        printf("segptr: %s\n", segptr);
}

removeshm(int shmid)
{
        shmctl(shmid, IPC_RMID, 0);
        printf("Shared memory segment marked for deletion\n");
}

changemode(int shmid, char *mode) 
{
        struct shmid_ds myshmds;

        /* Get current values for internal data structure */
        shmctl(shmid, IPC_STAT, &myshmds);

        /* Display old permissions */
        printf("Old permissions were: %o\n", myshmds.shm_perm.mode);

        /* Convert and load the mode */ 
        sscanf(mode, "%o", &myshmds.shm_perm.mode);

        /* Update the mode */
        shmctl(shmid, IPC_SET, &myshmds);

        printf("New permissions are : %o\n", myshmds.shm_perm.mode);
}

usage()
{
        fprintf(stderr, "shmtool - A utility for tinkering with shared memory\n");
        fprintf(stderr, "\nUSAGE:  shmtool (w)rite <text>\n");
        fprintf(stderr, "                (r)ead\n");
        fprintf(stderr, "                (d)elete\n");
        fprintf(stderr, "                (m)ode change <octal mode>\n");
        exit(1);
}
\end{verbatim} 
\nopagebreak\hrule 
\addvspace{\medskipamount}




\endinput
